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"2~D  Reconstruction  of  Injured  Brain  Structures" 

Narrative  Summary 

This  grant  supported  initial  research  and  development  efforts  to 
improve  methods  for  3-D  computer  graphics  reconstruction  of  damaged 
brain  tissues  in  living  subjects.  The  project  aimed  "to  produce  3- 
dimensional  images  and  provide  3-space  stereotactic  coordinates  for 
any  point  inside  the  skull  and  view  the  image  from  any  angle,  superim¬ 
pose  a  tailored  model  of  normal  neuroanatoroic  structures  on  the  image, 
produce  prototype  computerized  3-D  images,  and  identify  and  resolve 
problems  arising  in  developing  software  programs  using  representative 
CAT  Scans  of  VHIS  tapes". 

We  sought  to  analyze  some  of  the  clinically  more  intriguing  VHIS 
tapes  in  order  to  establish  whether  it  is  feasible  with  such  material 
to  obtain  accurate  and  reliable  3-D  reconstructions.  Scans  available 
were  taken  at  5  mm  intervals  using  routine  clinical  procedures  in  a 
series  of  volunteer  veterans  who  had  sustained  relatively  massive 
penetrating  wounds  of  the  head  during  the  Vietnam  War.  Two- 
dimensional  images  were  derived  from  data  on  VHIS  tapes  obtained  with 
a  1981-83-vintage  technology  GE-8800.  Reliable  3-D  reconstructions 
from  these  data  would  complement  traditional  methods  of  2-D  CT-Scan 
analysis  that  constitute  the  most  central  element  of  the  brain 
structure/function  study  program  inaugurated  by  William  F.  Caveness, 
M.D.  and  continued  by  Andres  M.  Salazar,  M.D.,  Colonel,  MC,  USA,  and 
his  colleagues. 

It  was  therefore  our  purpose  to  discover  which  parameters  of  den¬ 
sity  discrimination  would  be  most  likely  to  yield  discernable  boun¬ 
daries  between  skull  and  soft  tissue  (quite  easy),  between  nervous 
tissue  and  cerebrospinal  fluid  and  porencephalic  channels  (relatively 
more  difficult),  and  between  gray  and  white  matter  (really  infeasi¬ 
ble),  and  among  nuclear  masses  of  only  slightly  differing  densities 
(problemmatical ) .  This  leads  directly  to  consideration  of  the  problem 
of  accuracy  and  reliability  of  boundary  detection  among  these  various 
structures  and  in  relation  to  structures  involved  in  wounding  and 
their  consequences. 


Accuracy  and  Reliability  of  Boundary  Detection 

Accurate  3-D  reconstructions  would  closely  correspond  to  the 
actual  intracranial  contents,  enable  roughly  quantitative  interpreta¬ 
tion  of  defects,  and  provide  some  evidence  relating  to  the  dimensions 
of  the  functioning  remainder  of  brain  structures. 

The  only  conclusive  check  on  accuracy  of  computer  graphics  model¬ 
ing,  of  course,  is  obtainable  by  making  an  accurate  3-D  reconstruction 
following  death,  by  constructing  3-D  images  from  tissue  sections, 
postmortem.  This  is  a  procedure  which  the  Brain  Mapping  Project  is 
presently  in  the  process  of  developing.  The  remarkable  life-history 
profiles  established  by  the  VHIS  Project  makes  the  priority  for 


obtaining  autopsies  in  each  of  these  cases  take  on  extraordinary 
importance  for  the  analysis  of  brain  structure/function  relations. 
Evidence  could  be  obtained  sooner  and  more  easily  by  comparing  CT-Scan 
data  with  another  form  of  scanning,  e.g.,  magnetic  resonance  imaging 
(MRI).  Thus,  it  will  be  valuable  to  compare  CT-Scans  with  MR I  scans 
whenever  that  becomes  feasible.  Since  the  newer  technology  carries  no 
known  risk  to  the  individual  scanned,  a  comparison  of  past  CT-Scans 
with  MRI  scans  has  special  virtues  for  the  interpretation  of  both 
methods.  Even  without  that,  some  clues  can  be  obtained  simply  by  com¬ 
paring  successive  sections  of  the  same  scanning  run  because  neuroana¬ 
tomy  has  considerable  predictability  of  structural  continuity,  even  at 
5  mm  intervals.  Reliable  3-D  reconstructions,  of  course,  are  repeat- 
able  using  the  same  data  on  successive  runs. 


The  Need  for  Automatic  Boundary  Detection 

It  is  desirable  to  derive  the  2-D  images  automatically  (or  nearly 
automatically)  not  only  to  avoid  an  inordinate  expenditure  of  labor  by 
neuroanatomical  observers  that  is  otherwise  required,  but  also  to 
avoid  individual  observer  biases  and  letdown  from  distractions  and 
fatigue.  Automatic  boundary  detection  and  3-D  reconstruction  is  not 
only  potentially  more  accurate,  it  is  certainly  more  reliable,  objec¬ 
tive  and  essentially  indefatigable.  Automatic  boundary  detection  is 
necessary  to  reconstruct  a  sufficient  number  of  cases  to  establish  the 
database  required  for  making  comparisons  among  brain-injured  cases  and 
with  models  of  "normal  neuroanatomic  structures".  Furthermore, 
automatic  boundary  detection  is  essential  for  the  exportation  of  3-D 
reconstruction  methods  to  other  settings. 

Efforts  were  dedicated  to  creating  3-dimensional  images  and  pro¬ 
viding  a  suitable  strategy  for  obtaining  3-space  stereotactic  coordi¬ 
nates  for  any  point  inside  the  skull  and  views  from  any  angle  of  the 
brain  and  skull  images.  The  differences  in  density  between  bone  and 
soft  tissues  is  so  cliff-like  that  boundary  detection  is  not  a  problem 
whatsoever.  It  has  been  solved  by  a  number  of  programs  in  the  litera¬ 
ture  and  is  available  as  an  adjunct  to  several  commercially  available 
scanning  systems.  Stacking  of  skull  sections  to  make  three- 
dimensional  objects  is  likewise  no  problem.  Movement  of  the  head  can 
usually  be  easily  detected  and  at  least  partially  corrected  for. 

Some  filtering  and  arithmetic  adjustments  to  boundary  detection 
programs  can  sharpen  the  image  without  notable  distortions.  Recently 
Shankar  Chatterjee,  Assistant  Professor  of  Electrical  Engineering  and 
Computer  Science  at  UCSD  has  developed  a  boundary  detection  algorithm 
that  works  better  than  any  we  have  seen  in  operation  and  better  in 
principle  than  any  in  the  literature.  This  involves  a  bar-search  com¬ 
parison  of  contrasts  encountered  when  sweeping  the  bar  around  a  pixel 
in  the  vicinity  of  the  boundary.  Thus  when  the  bar  is  lined  up  along 
the  boundary  to  some  extent,  there  will  be  less  contrast  over  the 
extent  of  the  bar  than  when  it  is  lying  athwart  the  boundary.  The 
bar-search  can  economically  examine  for  contrast  and  define  the  boun¬ 
dary  in  this  way  as  it  moves  along.  Moreover,  since  it  operates  on 
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relative  contrast,  it  can  accommodate  large  degrees  of  difference  in 
steepness  (vividness  of  staining,  for  example)  along  different  parts 
|  of  the  boundary.  [Nevertheless,  when  the  boundary  is  washed  out  by 

!  low  level  noise  that  is  equivalent  to  boundary  definition  signals, 

even  the  best  software  is  unable  to  cope.] 


Super imposition  of  Model  of  Normal  Neuroanatomical  Structures 

A  second  major  contribution  undertaken  was  to  superimpose  a 
tailored  model  of  normal  neuroanatomic  structures  on  the  VHIS  case 
image.  This  we  did  by  projecting  onto  the  oscilloscope  either  the 
neuroanatomical  model  or  the  image  in  question.  The  image  produced  by 
the  Evans  and  Sutherland  Picture  System  2  could  then  maneuver  in  rela¬ 
tion  to  the  projected  image  and  sized  to  fix  approximately,  by  rota¬ 
tion,  translation,  and  scale.  Equivalence  of  the  two  images  is 
approximated  in  this  way  and  dimensions  estimated  in  accordance  with 
identifiable  E&S  parameters  and  arbitrary  or  (if  known)  actual  value 
scales  applied.  We  do  not  yet  have  a  "warping  algorithm"  to  accommo¬ 
date  3-D  warping.  Dr.  Fred  Bookstein,  one  of  our  long-standing  colla¬ 
borators,  at  the  University  of  Michigan,  says  that  under  his  guidance 
a  graduate  student  working  for  about  six  months  can  solve  this  prob¬ 
lem.  Thus  far  we  have  not  had  the  resources  to  apply  to  that  oppor¬ 
tunity.  For  the  present,  we  are  deferring  3-D  warping  in  favor  of 
learning  more  about  operating  experience  with  2-D  warping  in  a  wide 
variety  of  contexts.  Nonetheless,  we  shall  be  obliged  to  face  up  to 
the  requirement  for  3-D  as  well  as  2-D  warping.  So  far  as  we  know, 
there  is  no  existent  algorithm  that  can  be  applied  in  the  former  case, 
in  this  or  any  other  computer  graphics  applications. 

We  have  attempted  to  identify  and  resolve  problems  arising  in 
development  of  software  programs  as  applied  to  representative  CAT 
Scans  of  VHIS  tapes.  What  is  completely  refractory,  and  to  out 
knowledge  the  only  completely  refractory  problem,  is  that  relating  to 
the  paucity  of  contrast  in  the  raw  CT-Scan  data  with  respect  to  paren¬ 
chymal  tissue. 


Progress  and  Accomplishments  in  3-D  Reconstruction 


Simply  stated,  there  was  little  difficulty  in  creating  3- 
dimensional  images  of  skulls  from  which  3-space  stereotactic  coordi¬ 
nates  were  derived  for  localization  inside  the  skull  and  viewing  from 
any  desired  angle.  The  central  aim,  however,  was  to  develop  methods 
for  visualizing  and  making  explicit  3-dimensional  images  that  can  dis¬ 
tinguish  between  relatively  damaged  and  relatively  undamaged  parenchy¬ 
mal  tissues.  The  key  problem  is  the  "reconstruction  of  injured  brain 
(not  skull)  structures".  We  found  that  proper  reconstruction  of 
injured  brain  structures  was  not  possible  for  reason  of  the  narrow 
range  of  Hounsfield  numbers  that  depict  intracranial  structures,  using 
CT  Scans  of  that  vintage.  We  explored  several  different  approaches  in 
an  attempt  to  extract  reasonable  parenchymal  images,  but  to  little 
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avail.  Repeated  analyses  on  the  same  scan  section  gave  erratic  and 
unreliable  results.  Comparison  of  analyses  of  successive  sections,  at 
least  in  areas  of  chief  interest,  in  the  vicinity  of  lesions,  was 
inconsistent.  There  was  a  lack  of  expected  continuity  of  structure. 

)  * 

|  Each  method  tried  required  the  operator  to  exercise  prior 

i  knowledge  oT  neuroanatomyin  order  to  discriminate  boundaries  within 

[  the  tissue  regions  of  greatest  interest.  This  was  exactly  what  we  did 

I  not  want  to  have  to  depend  on:  prior  knowledge  of  what  "ought  to  be 

|  structurally  reasonable"  and  "what  might  be  interpreted  as  being 

i  altered"  in  some  spatially  definable  way.  Use  of  prior  knowledge  and 

|  of  subjective  criteria  did  not  yield  satisfactorily  objective  3- 

'  dimensional  reconstructions  for  purposes  of  charting  areas  of  damage 

|  that  intrude  into  or  displace  what  remains  as  residual  functional  tis¬ 

sue.  The  distinctions  between  relatively  obviously  damaged  regions 
and  apparently  relatively  undamaged  regions  was  not  only  arbitrary  but 
variable  when  reprojected  from  different  points  of  view. 

Ambiguities  remained  whether  we  used  the  "region  growing"  algo¬ 
rithm  developed  by  Dr.  Michael  Rhodes  or  software  previously  developed 
in  this  laboratory,  by  way  of  exploring  different  methods  for  boundary 
|  detection. 

It  would  be  unreasonable  and  unjustified  to  assume  an  ability  on 
the  basis  of  wEat  we  were  able  to  extract  from  the  digTtaT  readouts  oT 
tapes  provided^  to~5istinguish  among  the  possible  morphological  conse¬ 
quences  cTF  wounding  and  the  margins  of  damagecf~and/or  displaced  tis11 
sues  that  might  fc>e  functional ,~whether~Tunctioning  pathologically  or 
"normally" . 

We  asked  help  of  Dr.  Michael  Rhodes  because  of  his  extensive 
experience  in  CT-Scan  boundary  detection  and  in  extracting  2-D  data, 
but  he,  too,  was  unable  to  obtain  reliable  and  presumably  accurate 
boundaries.  We  asked  help  of  Dr.  Pavarti  Dev,  an  experienced  boundary 
detector,  principal  scientist  at  Contour  Medical.  The  interpretation 
of  tapes  sent  to  her  was  not  possible  in  her  experience.  Similarly, 
we  appealed  for  help  in  imaging  from  Gould  and  Silicon  Graphics  with 
similar  disappointment. 

There  is  no  problem  getting  images  of  inner  and  outer  tables  of 
skull  and  making  impressive  3-D  reconstructions  of  that  structure, 
including  bony  defects  in  the  VHIS  cases.  But  to  distinguish  paren¬ 
chymal  boundaries,  as  we  must  do  in  order  to  make  3-D  reconstructions 
of  injured  brain  structures,  defeated  us  all. 

We  made  some  crude  overlay  superpositioning  of  0035  whole  brain 
structures  onto  3-D  reconstructed  skulls,  using  Evans  and  Sutherland 
•  interactive  programs  to  position  the  two  structures  relative  to  one 

another,  one  being  projected  as  an  image  on  the  screen  while  the  other 
is  moved  about.  Yet  this  did  not  yield  clues  that  would  be  a  suffi¬ 
cient  guide  to  the  uncharted  parenchymal  neuroanatomy.  For  example, 
stereotactic  coordinates  are  of  no  avail  if  you  cannot  establish  boun¬ 
daries  of  defective  and  presumably  unaffected  brain  structures.  It 
was  even  impossible  to  detect  displacement  except  in  crude  terms  that 
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are  not  substantially  improved  over  what  is  visible  two-dimensionally . 
The  PI  invested  over  125  hours  total  in  painstaking  reconstruction 
using  the  Contour  Medical  CEMAX  1000,  under  the  guidance  of  Mr.  Wil¬ 
liam  Andrews  and  with  the  expert  clinical  advice  of  Dr.  Dennis  K. 
Bielecki.  Again,  it  was  not  difficult  to  obtain  reasonably  respect¬ 
able  looking  skull  parts,  including  defects  associated  with  injury. 
It  was  considerably  more  difficult  to  reconstruct  the  cerebral  ventri¬ 
cles  which,  even  in  the  best  instances,  looked  raggedy.  Where  poren¬ 
cephaly  obtained,  the  imaging  of  ventricular  defects  was  made  problem- 
matical.  Making  half-skulls,  with  brain  contents,  lesion  and  all, 
represented  within  the  cranial  vault  made  impressive  pictures  which  we 
sent  to  Dr.  Andres  Salazar,  but  there  could  be  no  reasonable  confi¬ 
dence  in  boundary  discrimination  —  it  looked  rather  like  bulky  amor¬ 
phous  shambles  in  different  shapes.  These  shapes  were  only  crudely 
equivalent  from  one  3-D  reconstruction  to  another  using  the  same  ori¬ 
ginal  data.  This  reflects  the  fact  that  with  Hounsfield  numbers  so 
close  together  the  boundary  detection  algorithm  is  responding  proba¬ 
bilistically  and  countour  are  without  well  defined  gradients. 

We  tried  making  "ring  stacks"  of  reconstituted  objects  in  order 
to  do  "editing"  in  the  form  of  3-D  sculpting  to  make  the  structures 
"look  more  reasonable."  But  this  approaches  idiocy  as  far  as  sensible 
and  useful  derivations  are  concerned. 


Conclusions ; 

With  the  33  cases  provided  from  the  VHISP  on  GE  8800  tapes,  we 
could  not  read  a  few  cases  and  could  not  read  some  parts  of  others, 
but  we  did  obtain  2-D  images  for  most  of  them  that  were  equivalent  to 
the  usual  CT-Scan  results.  We  successfully  reconstructed  bony  parts 
with  respect  to  all  cases  that  we  could  read.  The  only  difficulties 
encountered  in  some  instances  were  the  result  of  metalic  fragments 
which  gave  rise  to  deformations  of  the  beams  and  consequent  deforma¬ 
tions  of  contouring  in  the  vicinity  of  such  fragments.  But  we  never 
did,  in  any  case,  obtain  accurate,  reliable  boundaries  that  would 
enable  interpretations  to  be  made  as  to  what  were  the  margins  of  func¬ 
tional  tissue.  Since  that  was  the  principal  aim  of  the  project  we  can 
testify  that  skull  boundaries  and  stereotactic  coordinates  are  feasi¬ 
ble  but  that  quantitative  3-D  definition  of  the  extent  of  damage  and 
the  boundaries  of  presumably  functional  neuroanatomy  is  not  obtainable 
with  the  narrow  range  of  Hounsfield  numbers  available  with  that  vin¬ 
tage  of  CT-Scan.  Given  this  conclusion,  there  is  no  way  to  identify 
and  resolve  problems  arising  in  developing  software  programs  relative 
to  CAT  Scans  of  VHIS  tapes  —  rather,  the  problem  lies,  in  this  con¬ 
text,  in  hardware. 


